NORMAL FORMS FOR ORTHOGONAL SIMILARITY CLASSES 
OF SKEW- SYMMETRIC MATRICES 



DRAGOMIR Z. DOKOVIC, KONSTANZE RIETSCH AND KAIMING ZHAO 

Abstract. Let F be an algebraically closed field of characteristic different 
from 2. Define the orthogonal group, On(F), as the group of n by n matrices 
X over F such that XX' = In, where X' is the transpose of X and In the 
identity matrix. We show that every nonsingular n by n skew-symmetric 
matrix over F is orthogonally similar to a bidiagonal skew-symmetric matrix. 
In the singular case one has to allow some 4-diagonal blocks as well. 

If further the characteristic is 0, we construct the normal form for the 
On{^)-similarity classes of skew-symmetric matrices. In this case, the known 
normal forms (as presented in the well known book by Gantmacher) are quite 
different. 

Finally we study some related varieties of matrices. We prove that the 
variety of normalized nilpotent n by n bidiagonal matrices for n, = 2s -|- 1 is 
irreducible of dimension s. As a consequence the skew-symmetric nilpotent n 
by n bidiagonal matrices are shown to form a variety of pure dimension s. 
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1. Introduction 

In this note F denotes an algebraically closed field of characteristic not 2. By 
Mn{F) we denote the algebra of n by n matrices over F, by GL„(i^) the group 
of invertible elements of Mn{F), and by /„ the identity matrix of size n. For any 
matrix X, let X' denote the transpose of X. The orthogonal group, 0„(F), is 
defined as the subgroup of GL„(F) consisting of matrices X such that XX' = In- 

This paper is a sequel to [T] where the first and third author constructed tridi- 
agonal normal forms for symmetric matrices under the action of 0„{F). Here we 
continue this work and study the similarity action of 0„(F) on the space Skew„(i^) 
of all n by n skew-symmetric matrices over F. We show that each A E Skew„(i^) 
is orthogonally similar to the direct sum of blocks B which are either bidiagonal or 
4-diagonal. The latter is needed only if A has at least one pair of nilpotent Jordan 
blocks of even size. Each of the blocks B is similar to either a pair of Jordan blocks 
having the same size, s, and having eigenvalues A and —A, with s even if A = 0, or 
a single nilpotent Jordan block of odd size. 

If F has characteristic 0, we are able to choose the concrete representatives, 
i.e., normal forms for skew-symmetric matrices, see Theorem 13.21 In this case, the 
known normal forms for the 0„(_F)-similarity classes of nonsingular skew-symmetric 
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matrices, as presented in the well known book by Gantmacher |2j, are not bidiag- 
onal. Hence our new normal forms are simpler and may be better suited for some 
applications. At the end of Section [3] we pose three open problems. The proof of 
Lemma |3 . 1 1 uses some ideas of Givental and, due to its length, is given separately 
in Section 4. 

In the last section we study the variety i3„ H Mn of nilpotent n by n bidiag- 
onal matrices with I's along the lower diagonal (normalized nilpotent bidiagonal 
matrices), where n = 2s + 1. We compare Bn Ci Afn with the variety of normal- 
ized nilpotent tridiagonal matrices 7^ n Afn studied by Kostant in relation with the 
Toda lattice 0], and we show that Bn C\J\fn is irreducible of dimension s. These 
results also have consequences for the related variety of skew-symmetric nilpotent 
bidiagonal matrices, see CoroUarv 15.51 Finally we note that the coordinate ring of 
Bn n A/",! has an interpretation in terms of quantum cohomology of the flag variety 
SL„/B. 

The second author thanks the Univeristy of Waterloo for its hospitality during 
the second half of 2005. 

2. Bidiagonal and 4-diagonal representatives 

We first recall the following well known facts (see e.g. [6l Theorem 70] and [21 
Chapter XI, Theorem 7]). Gantmacher gives the proof over complex numbers but 
his argument is valid for any algebraically closed field of characteristic different 
from 2. 

Theorem 2.1. If two skew- symmetric matrices A, B £ Mn{F) are similar, then 
they are orthogonally similar. 

Let t be an indeterminate over F and let i denote one of the two square roots of 
-1 in F. Thus = _i. 

Theorem 2.2. A matrix A e Af„(_F) is similar to a skew- symmetric matrix iff the 
elementary divisors (t — A)* and (t -f A)* of A come in pairs if X ^ or s is even. 

We are interested in constructing normal forms of skew-symmetric matrices un- 
der orthogonal similarity. The case of complex matrices is classical and is described 
in Gantmacher's book j2j. Our objective is to construct a normal form which is 
almost bidiagonal, unlike the one given by Gantmacher. 

The general form of a skew-symmetric bidiagonal matrix is 






ai 





•• 

































-02 





as 

















-as 





























an-2 

















— a„-2 





a„- 

















— a„-i 






It is easy to see that if all a^'s are nonzero, then S is cyclic (i.e., its minimal and 
characteristic polynomials coincide). If all Ok 0, and a/c = 1 for even fc's then we 
say that this bidiagonal matrix is special. 
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Let s be a positive integer. We introduce a skew-symmetric nilpotent matrix Q^s 
of size 4,s having the elementary divisors , t^^ . The easiest way to understand 
the structure of this matrix is to take a look at one example: 



Q 



12 






-1 


i 






1 












—i 








-1 


i 






1 












—i 









1 

—i 



-1 


i 










—i 








-1 


i 


1 


-i 







1 



—I 

-1 





In order to define precisely these matrices, let us write Ep^g for a square matrix 
(of appropriate size, 4s in this case) all of whose entries are except the {p, g')-entry 
which is 1. Then we have 

Qis = X — X' , 

where X = Xi + X2 and 
Xi 



E. 



2p-l,2p 



X. 



2p,2p-l 



)• 



p=l 

2s 

,2p-2 

In the above example, the nonzero entries of X are enclosed in small boxes. 
We shall now prove the above mentioned properties. 

Lemma 2.3. The skew- symmetric matrix Q^g is nilpotent and has elementary 
divisors t^^ , t^^ . 

Proof. It is easy to verify that XX' = 0, and also that X1X2 = X2X1 = and 
X^''^^ = X^"*"^ = 0. The matrix XI resp. X| has exactly two nonzero entries 
namely resp. i^, i""^ and they are positioned in the middle of the first resp. 

last row. It follows easily that X'+'^ = 0, X^ = Xf + X|, and X'^X^ = 0. By 
using these relations and XX' = 0, we obtain that 



Qll = {X-X'Y 



= X 



2s 



x'x 



2s-l 



X'^X' 



x'^'-^x + x'^' 



i-iyx^x' = 0. 



As Q^s is nilpotent of rank 4s — 2, it follows that it has exactly two Jordan blocks 
each of size 2s. □ 

Remark 2.4. The matrix in the above lemma cannot be replaced by a bidiagonal 

skew-symmetric matrix S (of size n = 4s). Indeed assume that S is the bidiagonal 
matrix displayed above with superdiagonal entries ai, ...,a„_i. Since S must have 
rank n — 2, exactly one of the a^'s is 0. As it has elementary divisors t^^,t^^ we 
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conclude that a2s = 0. This contradicts Theorem 2.2 as the 2s x 2s block in the 
upper left hand corner has size 2s and only one elementary divisor t^^ . 

We can now state and prove one of our main results. 

Theorem 2.5. Every skew- symmetric matrix A G Mn{F) is orthogonally similar 
to the direct sum Ai® A2® ■ ■ ■ (B Am, where each block A^ is one of the following: 

(a) special bidiagonal of size 2s with the elementary divisors {t — A)*, (t + A)* 
and A 7^ 0; 

(b) special bidiagonal of size 2s + 1 with the elementary divisor 

(c) Qis, with the elementary divisors t^'^^t'^^. 

Proof. In view of Theorems 12.11 and 12.21 it suffices to construct A G Skew„ (F) 
having only one or two elementary divisors, as specified in the three cases of the 
theorem. 

Let us start with the case (a). Then n — 2s and the two elementary divisors are 
(i — A)'* and (i + A)-* with A 7^ 0. Let S = S{xi,X2, . . . ,Xs,y) he the skew-symmetric 
bidiagonal matrix whose entries on the first superdiagonal are the indeterminates 

xi,y,x2,y, ■ ■ .,Xs-i,y,Xs 

in this order. 

By using an obvious permutation matrix P, we can transform S'^ to obtain direct 
sum of two matrices of size s each: 

PS-^p-i = Si®S2. 

The matrix resp. iS'2 is the submatrix of 5^ which lies in the intersection of rows 
and columns having odd resp. even indices. These two matrices are tridiagonal and 
symmetric. Explicitly, the entries on the first superdiagonal of Si are 

xiy,X2y, ■ ■ ■,Xs-iy; 

and its diagonal entries are 

_2_2_2 _2 _2_2__2 

^IJ ^2 V T ■ ■ ■ 1 y T -^s V ■ 

The corresponding entries of S2 are 

X2y,x3y, ■ ■ -.xsy; 

and 

_2_2_2_2 _2 _2_2 

Xi y , X2 y , . . . , Xg_i y , Xg. 

We claim that the matrices Si — Si{xi, . . . ,Xs,y) and S2 — S2{xi, . . . ,Xs,y) 
have the same characteristic polynomial. Let X be the s x s matrix whose diagonal 
entries are xi, X2, ■ ■ ■ , Xg, those on the first subdiagonal are all equal to —y, while 
all other entries are 0. Then 5*1 — —XX' and S2 — ~X'X and our claim follows. 

Let us write the characteristic polynomial of the matrix X^y^Is — Si{xi, . . . ,Xs,y) 

as 

p{t) =t' + cit'-'^ + ■■■ + Cs-lt + Cs, 
where Ck ~ Ck{xi, ... ,Xs,y) is a homogeneous polynomial of degree 2k in the indi- 
cated variables. 

Since there are s-l- 1 indeterminates and F is algebraically closed, by |7i Theorem 
4, Corollary 5, p. 57] the system of homogeneous equations: 

Ck{xi,X2, ■ . . ,Xs,y) ^ 0, A: = 1, 2, . . . , s 
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has a nontrivial solution in F*+^, say (^i, $2, • • • , Cs; It follows that the matrix 

is nilpotent. We have r] ^ since otherwise this matrix would be diagonal and 
nilpotent, i.e., zero. The matrix 

is bidiagonal and the matrix A^/„ — A"^ nilpotent. As A is skew-symmetric, its 
eigenvalues must be A and —A, each with multiplicity s. 

Next we claim that A is special, i.e., that all £,k 7^ 0. We shall prove this 
claim by contradiction. So, assume that = for some k. Then the matrix 
}?rfls — 5'i(Ci, • • • ,S,s,v) breaks into direct sum of two blocks: the first Xi of size 
k and the second of size s — fc (if fc = s the second block is of size 0). Similarly, the 
matrix X^rj^Ig — £'2(^1, • • • , ^s, ??) breaks into direct sum of two blocks: the first X2 
of size k — 1 and the second of size s — k + 1 (ifA; = l the first block is of size 0). 
Since all these four blocks are nilpotent, their traces must be 0. Since 

tr(Xi) = kxw + {k~iw+{ei+---+ek-i), 
tr(X2) = {k-i)xw + {k~iw+{ei+---+ek-i), 

we deduce that A^r/^ = 0, which is a contradiction. 

Hence the matrix A/„ — A has rank n — 1. Consequently the elementary divisors 
of A are indeed {t - \y and {t + A)^ 

The case (b) will be handled by a similar argument. Let 5* = S{xi, X2, ■ ■ ■ ,Xs,y) 
be the skew-symmetric bidiagonal matrix whose entries on the first superdiagonal 
are the indeterminates 

xi,y,X2,y, ■ ■ . ,Xs-i,y,Xs,y 
in this order. The characteristic polynomial of this matrix has the form 

p{t) = ^2^+1 + Cif'-^ +■■■+ Cs-lt^ + Cst, 

where Ck = Ci.{xi, . . . ,Xs,y) is a homogeneous polynomial of degree 2k in the indi- 
cated variables. 

Since there are s + 1 indeterminates and F is algebraically closed, the system of 
homogeneous equations: 

Ck{xi,X2,...,Xs,y) = 0, fc = 1,2, . ..,s 

has a nontrivial solution in say {^1,^2, ■ ■ ■ ,^s,v)- It follows that the matrix 

^(^i, . . . , ^,s, v) is nilpotent. We claim that rj ^ 0. Otherwise this matrix would be 
nilpotent and semisimple, i.e., the zero matrix. 

Moreover we claim that each ^ 0. We prove this by contradiction. Thus 
assume that some = 0. Then the above matrix breaks up into direct sum of 
two blocks: the first of size 2k — 1 and the second of size 2(s — fc + 1). Both of 
these blocks must be nilpotent. However, the determinant of the second block is 
^2{s-k+i) _^ which gives a contradiction. 

Hence the matrix A = ?7~^>S'(^i, . . . , ry) is a special bidiagonal matrix having 
only one elementary divisor, f2*+^ . 

The case (c) is handled by the above lemma. 

□ 

Next we show that there are only finitely many special bidiagonal matrices in 
Skew„(F) having prescribed elementary divisors. 
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Theorem 2.6. If n = 2s is even and A 7^ 0, then there are at most 2^ si special 
bidiagonal matrices A G Skew„(i^) with elementary divisors {t — A)* and (t + A)'*. 
If n = 2s + 1 is odd, then there are at most 2*s! special bidiagonal matrices A G 
Skew„(i^) with the elementary divisor t'^'^'^^ . 

Proof. The existence of such matrices was proved in the previous theorem. We 
just have to show that the system of s homogeneous polynomial equations Ck = 
Ck{xi, . . . , Xs, ?/) = in s + 1 variables has at most 2*'s! solutions in the associated 
projective space. 

We claim that the number of solutions is finite. Otherwise the projective variety 
defined by this system of equations would possess an irreducible component, say 
X , of positive dimension. Consequently, the intersection of X with the hyperplane 
y ~ would be non-empty. On the other hand we have shown in the proof of the 
previous theorem that there are no nontrivial solutions with y — 0. Thus our claim 
is proved. 

Now the assertion of the theorem follows from Bczout's Theorem (see [71 Chapter 
IV, §2]). □ 

The non- uniqueness of special bidiagonal matrices in Skew„(i^), with specified 
elementary divisors, prevents us from obtaining a genuine normal form. 



3. Normal forms in characteristic zero 

From now on we restrict our attention to algebraically closed fields F of charac- 
teristic 0. In this case we find very simple normal forms for orthogonal similarity 
classes of skew-symmetric matrices. In the case of nonsingular matrices, this normal 
form is bidiagonal. 

Let n = 2s -I- 1 be odd and let i?„ G Skew„ {F) be the bidiagonal matrix whose 
consecutive superdiagonal entries are 

^/s, i, \/s — 1, iV2, Vs — 2, iVS, . . . , V^, iV s — 2, V2, iVs — 1, 1, i^/s- 

For instance, 



i?7 = 






V3 














" 


V3 





i 

















~i 





V2 

















-V2 





iV2 

















~iV2 





1 

















-1 





iV3 























It is not hard to show that the matrix R„ is nilpotent. The proof is similar to 
the proof of ^ Proposition 3.3] and we shall omit it. 

For even n — 2s, let P„ G Skew„(i^) be the bidiagonal matrix whose consecutive 
superdiagonal entries are 



' k{n - k) 
{n - 2fc)2 - 



1 < fc < 
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Note that for k ^ s the number is a positive rational number while 
We may assume that as — si. For instance, 





" 


V3 











" 




-V3 





2V6 











1 





-2V6 





9i 








3 








-9i 





2^6 
















-2V6 





^/3 
















-V3 






Lemma 3.1. Let n 

(P„2 - = 0. 



2s be even and let Pn be the matrix defined above. Then 



The proof of this lemma is somewhat long and complicated and will be given in 
the next section. 

Since one has to extract square roots, there is a built in non-uniqueness in the 
definition of the matrices P„ and i?„. Hence they should be viewed as defined only 
up to the choice of these square roots, or equivalently, up to the action of the group 
of diagonal matrices with the diagonal entries ±1. By abusing the language, we 
shall refer to this type of non-uniqueness as the choice of signs. 

Since all entries on the superdiagonal of P„ are nonzero, we conclude that the 
elementary divisors of P„ are {t — 1)* and {t + 1)". 

As a consequence, we obtain the following theorem. 

Theorem 3.2. Let F be an algebraically closed field of characteristic 0. Then any 
.skew- symmetric matrix A € Mn{F) is orthogonally similar to the direct sum of 
blocks of the following types: 

(a) XPm, m even, X ^ 0; 

(b) Qm, fn divisible by ^; 

(c) Rm, ra odd. 

This direct decomposition is unique up to the ordering of the diagonal blocks and 
the choice of signs inside the blocks of type P™ and . 



We can now derive some interesting combinatorial identities from Lemma 13.1 
We fix a positive integer s, set n = 2s, and define the coefficients 

k{n — k) 



(3.1) 



kez. 



[n - 2fc)2 -V 
Note that then (3k — Pn-k is valid for all integers k. 
Corollary 3.3. For 1 < k < s we have 

l<il<i2«-<«fc-l«ifc<n-l 

where i ^ j means that j — i > 2. 

Proof. The characteristic polynomial of the matrix P„ is 



fit) = e - Clt 



ri-2 



C2t 



n— 4 



where 



Cfc 



E 

l<il<Cj2<C-<jfc-i<gifc<n— 1 



-••• + (-irc. 
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By Lemma [STTl P„ has elementary divisors (< — 1)", (t+ ly. Hence, f{t) — — ly 
and the identities stated in the coroUary follow. □ 

We end this section by proposing three related open problems. 

Problem 1 Find a direct proof of the combinatorial identities stated in the above 
corollary. 

Problem 2 If has characteristic zero and n = 2s is even, then there are exactly 
2*s! special bidiagonal matrices in Skew„(_F) having elementary divisors {t — 1)", 
{t + ly . (We have verified this claim for s < 4.) 

Problem 3 If F has characteristic zero and n = 2s + 1 is odd, then there are 
exactly 2*s! nilpotent special bidiagonal matrices in Skew„(i^). (We have verified 
this claim for s < 4.) 



4. Proof of Lemma [3TT] 

We recall that n = 2s is even and that the /J^'s are defined by the formula (|3.ip . 
Clearly, the matrix P„ is similar to 



X = 






Pi 











-1 
















-1 























Pn- 











-1 






The matrix X'^ has zero entries in positions with i + j odd. Hence X^ 

is permutationally similar to the direct sum of two s x s matrices Y and Z . The 
matrix Y (resp. Z) is the submatrix of X^ occupying the entries in positions {i,j) 
with i and j odd (resp. even). The matrices Y and Z are similar since Y = UV 
and Z = VU, where 



and 



U = 



V 



I3i 














-1 
















-1 

































-1 


I3n-1 


-1 
















-1 
















-1 

















-1 
















-1 



It remains to prove that the matrix Y is unipotent, i.e., the matrix Ig — Y is 
nilpotent. For the reader's convenience, let us display the matrix Ig — Y: 
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-1 







-1 





1 + /?4 + /35 












1 + l3n-4 + Pn-3 — Pn-3,Pn-2 

-1 1 + /3„_2 + /3„_1 



The proof of this last fact is quite intricate, it uses Givental's proof of nilpotency 
of some special tridiagonal matrices constructed from a finite quiver (see his paper 
[5]). For the reader's convenience, we include more detailed proof. 

Denote by T the infinite square grid with vertex set and orient each horizontal 
edge to the right, (i, j) ^ (i + 1, j), and each vertical edge downward, (i, j + 1) ^ 
(i, j). To this horizontal resp. vertical edge we assign the weight 

2i{2i + l) 



resp. 



(2i-2j + l)(2i-2j + 3) 
2j(2j - 1) 

Vi 



(2z-2j-l)(2z-2i + l)- 
It is easy to verify that 

(4.1) Uij + Vij^l = Mj-lj + V^J, UijUij = Uij + lWi+lj, 

i.e., for each vertex («,j) G F the sum of the weights of the two incoming edges is 
the same as for the two outgoing edges and, for each small square of F, the product 
of the edge weights along the two oriented paths of length 2 are equal (see Figure 
1). Note that mqj = for all j's and Vi^ = for all i's. 



J + 1 



J-1 



i-l 



• ^ 1 


1 ^ 1 1 

1 ^ ( 1 




1 



i + 1 



Fig. 1: A portion of F 
For each integer d > 1 define two square matrices of size d + 1 
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Ud 



Ud,i 

1 Ud-1,2 







1 Ud-2,3 














ui,d 
1 



and 



Vd = 



-1 Vd-_ 







'1 Vd- 







The matrices 

-Ud,l 

-1 




UdVd 



and 



Ud,lVd,l 
Vd.l - Ud^ 

-1 



1,2 



VdUd^ 



Vd,l 



3 
] 



Ud-1.2Vd 
Vd-1,2 - Ud 






Vl,d 

-1 



1,2 
2,3 



V2,d-1-Ui^d Ul,dVl.d 
-1 Vid 



-1 Wd-1, 2 - Ud-1,2 U<i-2,3W<i-l,2 

-1 Vd-2,3 - Ud-2,3 



Vim 



-uiM 
-1 



are tridiagonal. 

One uses induction on d > 1 to prove that UdVd is nilpotent. The case d = 1 is 
trivial to verify. Now let d > 1. As UdVd and VdUd are similar, it suffices to prove 
that VdUd is nilpotent. This is the case iff the submatrix obtained from VdUd by 
deleting the last row and column is nilpotent. The equalities (|4.1|) imply that this 
submatrix is equal to Ud-iVd-i- Hence we have shown that all matrices UdVd are 
nilpotent. 

Finally it remains to observe that Is — Y = Us-iVs-i, which can be easily verified 
by using the definition of the weights Uij and Vi,j. 

5. Some related varieties of matrices 

Let Afn be the nilpotent cone in Mn{F), i.e., the variety of all nilpotent matri- 
ces in this algebra. Denote by 5„ the subspace of Af„(F) consisting of all skew- 
symmetric bidiagonal matrices, by %i the affine subspace of Mn{F) consisting of 
all tridiagonal matrices A = [aij] with fli+i^i = 1 for i = l,...,n— 1, and by Bn the 
affine subspace of 7^ consisting of the matrices having zero diagonal. Finally, let 



s: c 5„ 



cr„. 
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be the open subvarieties consisting of the matrices having nonzero entries along the 
first super-diagonal. Note that any matrix in S* is similar to a unique matrix in 
S*, and the resulting map S* ^ is a 2"~^~fold covering. 

Unless stated otherwise, we assume from now on that n = 2s + 1 is odd. We 
are interested in the intersections Bn f^Mn and Sn H A/'n and their coordinate rings. 
We will relate these to a closed subvariety Vs C F"^^ to be defined shortly. We 
introduce first three maps 



s+l, 



A. : F" 



r„ B : F 



by defining 



^I(P) 



-Pi 
1 



P1P2 

-P2 - P3 
1 



P3P4 
-P4 - P5 



ji-1 



Pn-2Pn-l 
-Pn-1 



A2{P) 



-Pl - P2 
1 



P2P3 
-P3 - P4 
1 



and 



B{p) 



where p = (pi, . . . ,p„-i) G i^" 



-Pi 

1 



PiPb 
-P5 - P6 



-P2 




Pn-3Pn-2 
-Pn-2 -Pn-1 



-Pn~l 




Proposition 5.1. The characteristic polynomials 0} Ai(p), ^2(p) o,nd B{p) are 
related as follows. 



(5.1) 
(5.2) 



det(t/,+i- Ai(p)) = t dct{t Is - A2{p)), 
det(t/„-S(p)) = i det(t2/, -A2(p)). 



Hence, if one of the matrices Ai{p), A2{p), B{p) is nilpotent, so are the other two. 
Proof. Following Givental [3], we define the matrices (of size s + l) 



U{P) 



-Pi 
1 



~P3 
1 



-P5 



-Pn-2 

1 
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and 



1 ~P2 

1 -pi 
1 



1 



Then [/(p)V'(p) = ^i(p) and 

l/(p)[/(p) = 

where w is a row vector. Consequently, 



A2(p) 

w 



det( t h+i - Ai(p)) = dct( t Is+i - C/(p)y(p)) 

= det(i - V{p)U{p)) =tdct{tls - ^2(p)). 



It is easy to verify that there is a permutation matrix n„ G Mn{F) such that 

n„s(p)2n-i 



^i(p) 
A2{p) 

holds for aU p e F""^ So, by using (ISTTj) . 

det(i /„ - B(p)2) = det(t - Ai(p))det(t - yl2(p)) 

= tdet(t/, - A2(p))2. 

On the other hand it is easy to see that det(t /„ — B{p)) = dct(t /„ + B{p)) and 
so 

det(t /„ - B(p))2 = det(t2 /„ _ Bip f) = det{t^ h ~ ^2(p))'. 

Therefore det(i /„ - B{p)) = ±tdel{t^ Is - A2(p)) which after comparing the 
leading coefficients implies (|5.2p . □ 

The non-constant coefficients of the characteristic polynomial of -B(p), or equiv- 
alently of Ai{p) or A2{p) define a variety (as we will see, reduced) inside F'^'^ which 
we denote by Vs- In other words. 

Define also V* = n (F*)""! and note that the map B induces isomorphisms 
V, ^ Bn n Nn and V* ^ S,* n Nn- 

Let ai : Vs ~> T^+i n Us+i and : V* ^ 7^*^^ n Ns+i be the maps induced by 
Ai, and define similarly the maps a2 and aj. 

Proposition 5.2. V* is a smooth, irreducible variety of dimension s and a\ is an 
open inclusion. 

Proof. By Kostant's work on the Toda lattice (see [U Theorem 2.5]) the scheme- 
theoretic intersection Tg^iHAfs+i defines a smooth, irreducible variety of dimension 
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s. In fact it is isomorphic to an open subset of F 
that aj is an open embedding. A matrix 

-ai bi 

1 -02 &2 



It suffices, therefore, to show 



er;+inA4+i 



-as bg 

1 -a-s+i 

Ucs in the image of a* precisely if the denominators in the continued fraction 
expansions 



Pi=ai, P2 



bi ^ bi 

= —, P3 = a2-\ , 

oi ai 



Pi = 



b2 



a2 + 



P5 = a3 + 



b2 



a2 + 



up to 



Pn-l 



are all nonzero. This clearly defines an open subset of T*_^-^ (iNs+i- 
formulas for the pi define an algebraic inverse from this open set to V* 



The above 
□ 



We have therefore proved that B* fl J\fn is a smooth, irreducible variety of di- 
mension s. 

Denote by As the set of subsequences A = (Ai, . . . , A^) of (1,2,..., 2s) with k 
even, < fc < 2s, and such that Xi = i (mod 2) for each i. Let p = {pi, ■ ■ ■ ,P2s) € 
Vs and let Ap = (Ai, . . . , Afc) be the increasing sequence of indices such that 

PAi = • • • = PAfc = 

and all other coordinates pi are nonzero. Since nilpotent matrices have zero deter- 
minant, it is easy to see that Ap G Ag. Consequently, we have a map cr : — > A^ 
defined by o'(p) = Ap. Wc denote by the fibre of a lying over the point A S Ag. 
This gives a set-theoretic partition 

(5.3) V. = ]J V^. 

AeAa 

These fibres are in fact smooth subvarieties. For instance, for the empty sequence 
€ As we have Vf = V*. Moreover, wc have the following description of the fibres. 

Proposition 5.3. For A = (Ai, . . . , Afc) € As, k = 2r, we set Ao = 0, Afe+i = n 
and Si = {Xi — A,_i — l)/2, 1 < i < fc -I- 1. Then 

v,^= n 
i<i<fc+i 

where, by convenUon, Vq is the variety consisting of a single point. In particular 
Vg is a smooth variety of dimension s — r. 

Proof. Let p G V^. Then the characteristic polynomial of B{p) agrees with the 
characteristic polynomial of the block matrix 

Ml 

M2 



Mk+1, 
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where 

-PA._1 + 1 

1 -P\,-i+2 

1 

The matrix M is nilpotcnt if and only if each of the Mi is nilpotent. Therefore 

Pi ■= {P\-i + l,P\i-i+2, ■ ■ ■ ,P\i-l) 

lies in V*. . The resuhing map 

- n K 

i<i<fc+i 

P ^ (Pl,P2:---,Pfe+l) 

is clearly an isomorphism. □ 
Theorem 5.4. is an irreducible variety. 

Proof. Vs is defined as the intersection of s hypersurfaces, the zero sets of the 
coefficients of t*^, A: = 0, . . . , s — 1, of the characteristic polynomial of A2{p). Con- 
sequently, each irreducible component of Vs has dimension at least s. On the other 
hand, Proposition 15.31 implies that these dimensions are at most equal to s. Hence, 
Vs is an equidimensional variety of dimension s. It follows that is irreducible. □ 

Corollary 5.5. 5„nA/'n and S*nJVn are varieties of pure dimension s. Moreover 
S* n Afn is smooth and an open dense subset of 5„ D Afn ■ 

Proof. We have a well-defined map : 5„ n 7V„ Vs which takes a nilpotent skew- 
symmetric bidiagonal matrix with entries (ai, . . . , a„_i) along the upper diagonal, 
to an element (— a^, . . . , —a^_i) G Vs- This map restricts to a covering over each of 
the fibers , which implies that 5„nA/'n is reduced and any irreducible component 
has dimension at most s. However, as for V^, any irreducible component of 5„nA/'„ 
has dimension at least s. Therefore iS„nA/'n is of pure dimension s. As a consequence 
the s-dimensional fiber, S* Afn, is an open dense subset of Sn n Afn. Moreover, 
since S* JVn is a covering space of V* , it is a smooth variety. □ 

The problem of finding "nice" bidiagonal normal forms for orthogonal similarity 
classes of skew-symmetric matrices is identical to exhibiting "nice" representatives 
of the intersection 5„ H A/'n. In connection with this it is important to determine 
precisely the irreducible components of Sn HA/'n. If n = 3 it is easy to check directly 
that S3 n As has two irreducible components. 

Problem 4 Prove or disprove the following assertion. For n > 3 odd, the variety 
5* n JVn is connected, and 5„ n JVn is irreducible. 

Remark 5.6. The above considerations are for varieties of n x n matrices where n 
is odd. For even n = 2s we remark that B2S H JV2S is a variety of pure dimension 
s — 1, but no longer irreducible if s > 1. Namely it is easy to see that one has a 
decomposition into s components, 

B2snM2s = U^J' 
i=i 
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where is defined by the vanishing of the (2j — 1)''* entry on the upper diagonal. 
Each of these components is isomorphic to a product, Xj = Vj-i x Vs-j, and hence 
is irreducible and (s — l)-dimensional. 

For the even-dimensional skew-symmetric case the scheme-theoretic intersection 
S2s n is not reduced, so we choose to consider 823 H N2S as an intersection 
of algebraic sets. The resulting variety then again decomposes into a union of s 
subvarieties Xf^™^ where the definition of Xf^™ is analogous to that of X, above. 
Moreover, the component X^^™ is again isomorphic to a product, 

(5.4) (52J-1 n N2j-i) X {S2S-2J+1 n 7V2..-2J+1), 

and Corollary 15.51 implies that it has pure dimension s — 1. Hence H A2S has 
pure dimension s — 1 . 

The number of irreducible components of H A/'2s, when s > 1, depends on 
the numbers of irreducible components of the subvarieties (j5.4p . Namely, since 
iSs n As has two irreducible components and thus Xf^™ and Xf^^ have at least 
two irreducible components each, the variety 52s H J\f2s must have at least s + 2 
irreducible components. 

Explicitly we have the following. In the case s ~ \ the algebraic set ^2 n N2 
consists of a single point. For s — 2 there are precisely s + 2 = 4 irreducible 
components in 54 n A/'4, obtained as described above. The case s = 3 is exceptional 
with 6 irreducible components. This is because the variety X^™ is isomorphic to 
(iSs n A/jj) X (iSs n As) which has 4 irreducible components, and it can be checked 
that Xf^™ and Xf^™ are irreducible. For s > 4, if the assertion in Problem 4 is 
true then the variety 52s H A/'2s has precisely s + 2 irreducible components. 

The results in this section may be compared with the normalized tridiagonal and 
the tridiagonal symmetric cases. 

In the normalized tridiagonal case 7^ n Mm was shown to be irreducible (of di- 
mension TO — 1) by Kostant [5]. This variety is of particular interest as its coordinate 
ring has another interpretation as the quantum cohomology ring qH* (Sl^m/ B, F) 
of the flag variety Shm/B. In this context, the coordinate ring of our variety 
Bn n Afn can also be interpreted as the quotient of the quantum cohomology ring 
qH*{SLn/B, F) by the ideal generated by the Chern classes of the tautological line 
bundles, Xi = ci{Li). Thus Theorem 15.41 implies that 

qH*{SL„/B,F)/{xu...,Xr,) 

is an integral domain (for odd n). 

The variety of symmetric nilpotent m x m matrices is also equidimensional, as 
can be shown without much difficulty, and is irreducible precisely if its intersection 
with the regular nilpotent orbit (which is smooth) is connected. 
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